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Abstract 

The note begins with a short story on seeking for a practical suffi¬ 
ciency theorem for the uniqueness of time-continuous Markov jump pro¬ 
cesses, starting around 1977. The general result was obtained in 1985 for 
the processes with general state spaces. To see the sufficient conditions 
are sharp, a dual criterion for non-uniqueness was obtained in 1991. This 
note is restricted however to the discrete state space (then the processes 
are called Q-processes or Markov chains), for which the sufficient condi¬ 
tions just mentioned are showing at the end of the note to be necessary. 

Some examples are included to illustrate that the sufficient conditions 
either for uniqueness or for non-uniqueness are not only powerful but 
also sharp. 
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Let E be a countable set with elements A matrix Q = (q l] : i,j e 

E ) is called a Q-matrix if its non-diagonals are nonnegative and YjeE Qij ^ 0 
for every i £ E. Throughout this note, we restrict ourselves to the special 
case that the Q-rnatrix is totally stable q, := —qa < oo and conservative 
qi = Yj^i Qij f° r every i e E. It is called bounded if sup ieE qi < oo. For a 
given Q-matrix Q = {qij) on E, a sub-Markovian semigroup {P(t) = ( Pij(t ) : 
i,j £ E)}t ^o is called a Q-process if 

= Q (pointwise). 
t=o 

The Q-processes may not be unique in general, but there always exists the 
minimal one, due to Feller (1940) [17; Theorem 1], denoted by P mm (t) = 
() : i,j £ E). For more than half-century ago, some criteria for the 
uniqueness were known. 

Theorem 1 The Q-process is unique (equivalently, the minimal process P mm (f) 
is not explosive) iff one of the following equivalent conditions holds: 

(Cl) YjeEPfj m (t) = 1 f° r ever Y i £ E and t ^ 0. 
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(C2) 2* ! q-' = go, Pj-a.s., where r n is the nth jump time of the minimal 

process {X mm (t) : t ^ 0} corresponding to 
(C3) The equation 

(A I — Q)u = 0, 0 ^ u ^ 1, (1) 

has only zero solution for some (equivalently, for all) A > 0. 

Criterion (Cl) goes back to Feller (1940) [17]. Criterion (C2) is due to Do- 
brushin (1952) [16]. Criterion (C3) is due to Feller (1957) [18] and Reuter 
(1957) [23]. Refer also to Chung (1960) [15; Part II, §19, Theorem 1], or [19; 
Chap. 3, §2, Theorems 3 and 4], 

The earlier Criterion (Cl) often requires a further effort in practice, rather 
than a direct application. In particular, the proof of the powerful sufficiency 
theorem (Theorem 3 below) is based on it. 

Criterion (C2) is effective in some cases. For instance in the simplest case 
that M := sup ie EQi < oo, since 


2 

n =1 


? .Ymi n( Tn ) 


> 


z 

n =1 


M~ 


= oo, 


we obtain the uniqueness of the processes. For pure birth process (i.e., g* ,*+1 > 
0 and qtj = 0 for all j i, i,j ^ 0), Criterion (C2) says that the process is 
unique iff 


00 


z 

n =1 


i 


Qn,n +1 


= 00 . 


( 2 ) 


Besides, if the minimal process is recurrent, then the term q^ 1 will appears in¬ 
finitely often in the summation, hence the process should be unique according 
to the criterion. 

Criterion (C3) is more effective once equation (1) is solvable. More pre¬ 
cisely, it is the case if the exit boundary consists at most a single point, for 
instance the pure birth processes, the birth-death processes or more general 
the single birth processes (i.e., for j > i ^ 0, qij > 0 iff j = i +1; for 0 ^ j < i, 
qij is nonnegative but free). We will come back this story soon. 

However, the next model stopped our study for several years at the begin¬ 
ning of the study (1977-1978) on non-equilibrium particle systems. To state 
our model, we use operator H instead of the matrix Q: 


Mf(i) = J] qij{fj - fi), i e E. 

jeE 


Of course, in this case, Q/ = Qf. For a Markov chain on a countable set E, 
by a transform, one often assumes that E is simply the set Z + = {0,1, • • • }. 
However, such a transform ignores the original geometry of E and may not 



Practical Criterion for Uniqueness 


3 


be convenient in multidimensional case. To state our model, we need some 
notation. Let i = (i u : u e S) and define its updates i u - and i u,v as follows: 


,;U+ 



W = U 

w =£ u, 


i u ’ v = < 

L W ' 


r i u - 1 

In, + 1 


w = u 
w = v 
w ¥= u, v, 


weS. 


Example 2 [Schlogl's second model] Let S be a finite set and E = 7L S + . Define 
a Markov chain on E with operator 

n /(*) = S { b (*«)[/(* W+ ) ” /(*)] + «(*«)[/(* w “) “ /(*)]} 

ueS 

+Y J i uP( u , v )[f {i U,V ) ~ /(*)]> i = (i u :ue S) e E, 

IL,V 

where ( p(u,v ) : u,v 6 S) is a “simple” random walk on S, and 

b(k) = Po + /3 2 k(k - 1), A), /3 2 > 0, 
a(k) = 5\k + 5^k{k — 1 )(k — 2), > 0. 

Here in the first sum of in each vessel u, there is a birth-death process 
with birth rate b(k) and death rate a(k), respectively. This is called the 
reaction part of the model. The reactions in different vessels are independent. 
In the second sum of H, a particle from vessel u moves to vessel v. This is 
called the diffusion part of the model. Thus, it is actually a finite-dimensional 
reaction-diffusion processes. Replacing the finite S with S = r L d , we obtain 
formally an operator of infinite-dimensional reaction-diffusion process which 
is a typical model from the non-equilibrium statistical physics. Even though 
the large systems are quite popular today, in that period, it was rather unusual 
to study such a non-equilibrium system. Our original program is to rebuild 
the mathematical ground of non-equilibrium statistical physics (cf. [9; Part 
IV]. An earlier paper on this topic appeared in 1985 [2]). For this, the model 
is meaningful only if it is ergodic in every finite dimension. Thus, the finite 
dimensional model consists the first doorsill of our program. 

In 1983, the author and Yan [26], using a comparison technique, overcame 
this doorsill, based on a systemic study on the single birth processes. To which, 
we obtained explicit criteria not only for uniqueness but also for ergodicity and 
so on. This goes back to [26, 3]. Refer to [9] for updates and to [13] for a 
unified treatment. After two more years, using an approximating approach, 
we obtained a powerful sufficiency theorem as stated below. 

Theorem 3 [Uniqueness criterion] Let Q = ( qij) be a Q-matrix on a coun¬ 
table set E. Then the corresponding Q-process is unique iff the following two 
conditions hold simultaneously. 
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(Ul) There exist E n \ E as n \ oo and a nonnegative function tp such that 

SU P ie En Qi < 0° and lim n _ >00 inf ^ En Vi = oo. 

(U2) There exists a constant cel such that Qtp ^ cip- 

Certainly, for Schlogl’s model for instance, in condition (U2), it is more 
convenient to use f lip instead of Qtp. Besides, an important fact should be 
very helpful in practice: if v satisfies the conditions with c ^ 0 , then so does 
M + ip for every constant M ^ 0. In particular, a local modification of Q does 
not interfere the conclusion. 

From [9; Parts I and II], it is now clear that a large part of the theory 
of Q-processes can be generalized to the so-called Markov jump processes 
on general state space. To save the space, we will not really go to the last 
subject but it is worth to mention the extension. We now use the codes “GS” 
and “DS” to distinguish the “general state space” and the “discrete state 
space”, respectively. The sufficient part of the last theorem first appeared in 
[3; Theorem 2.37 (GS)] and [4; Theorem 16 (GS)]. Because it is regarded as one 
of the author’s favourite contributions to the theory of Markov jump processes, 
this result was then introduced several times in the author’s publications: [ 6 ; 
Theorem 1.11 (DS)], [12; Theorem 3.9 (GS)], [9; Theorem 2.25 (GS)], [ 8 ; 
Theorem 2.1 (DS)], [10; Theorem 9.4 (DS)], and [11; Theorem 2.9 (DS)]. 

Theorem 3 is often accompanied in the publications just listed by the next 
simpler result. 

Corollary 4 Suppose that there exist a function p> ^ q and a constant c e R 
such that Qtp ^ c<p on E. Then the Q-process is unique. 

Proof. Set E n = {i e E : q t ^ n}. If M := sup ieE qi < oo, then for large 
enough n, we have E n = E and so inf^_E re qk = oo by standard convention 
inf 0 tp = oo. In this case, condition (U2) is trivial with tp = 1 + M. If M = oo, 
then inffc^E n % ^ n — > oo as n — > oo. Combining this with (U2), the conclusion 
follows from Theorem 3. □ 

Corollary 4 is almost explicit since one can simply specify p = 1 + q. This 
enables us to use it easier in practice. However, such a specification makes the 
assumption becomes a little stronger. We will come back this point later. 

Let us make some remarks about the conditions in Theorem 3. Condition 
(U2) is a relax of the equation in (1): finding a solution to an inequality is 
easier than finding a solution to the corresponding equality. Criterion (C3) 
says that there is only trivial bounded solution to the equation (1). Con¬ 
versely, if a solution of the equation is fixed at some point, say 9, such that 
<P 0 = 1, then the solution tp should be unbounded. This leads to the condition 
lim n _ >00 inffc££„ tpk = oo in (Ul). Using this idea, we prove that the assump¬ 
tions in Theorem 3 are necessary for single birth processes [9; Remark 3.20]. 
The reason we allow some subset of E n to be infinite is to rule out some region 
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of E. on which supj eEn qt < oo. The key in the proof of this result is an eco¬ 
nomic approximation by bounded Q-processes. Certainly, the necessity shows 
that the assumptions of the theorem are sharp, and is valuable as illustrated 
by [4; Theorem (25)]. However, it does not mean that the inverse of the condi¬ 
tions in Theorem 3 can be used in practice to show the non-uniqueness of the 
processes. Hence, we went to an opposite way proving the following criterion 
[9; Theorem 2.27 (GS), its proof in 2 nd edition uses Lemma 5.18 rather than 
Lemma 5.15]. 

Theorem 5 [Non-uniqueness criterion] For a given Q-matrix Q on a countable 
set E, the Q-processes are not unique if for some (equivalently, for all) c > 
0, there is a bounded function ip with sup keE^k > 0 such that Q<p ^ c<p. 
Conversely, these conditions plus <p ^ 0 are also necessary. 

We remark that three results (Theorems 3, 5 and Corollary 4), we have 
talked so far are specialized from their original case in GS to the one in DS. 
Theorems 3 and 5 are somehow the extensions of Criterion (C3) in two opposite 
directions. As we will see soon that the extended theorems are much effective 
than the original Criterion (C3). Using two opposite sufficiency results instead 
of a single criterion is often meaningful. For instance, for recurrence, we have 
a criterion [9; Proposition 4.21] which is accompanied with more practical 
criteria [9; Theorems 4.24 and 4.25] for the recurrence and transience, respec¬ 
tively. As a companion to [9; Theorem 4.25], refer to [22; Theorem 8.0.2] and 
[20; Proposition 1.3] or more recent criteria. Next, for ergodicity and noner- 
godicity, refer to [9; Theorem 4.45(1)] and [21; Theorem 1], respectively. For 
various stability speeds/principal eigenvalues, in [10], we have not only the 
classical variational formula, but also dual variational formulas to describe 
their lower and upper bounds, respectively. 

It is interesting that there is now a direct way to prove the necessity of 
Theorems 3 in the context of DS based on a recent result by Spieksma [25]. 

Theorem 6 Everything is the same as in Theorem 3 except (Ul) is replaced by 

(Ul)' In the original (Ul), assume in addition that each E n is finite and ignore 

“sup ieEn Qi < 00 ". 

It is now the position to illustrate by examples the power of our results 
and compare conditions (Ul) and (Ul)'. 

The next two examples show that in Theorem 3, the condition “lim^^oo P>n 
= go” is not necessary, which is however necessary in a criterion for recurrence 
used in the proof of Theorem 6 (see its proof below). 

Example 7 Let E be a countable set and Q = (r/,y ) be a bounded conservative 
Q-matrix on E. Then assumptions of Theorem 3 hold but its test function ip 
can be bounded. 
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Proof. (a) Simply set E n = E (may be infinite) for every n ^ 1 and p % = l. 
Then it is obvious that 0 = Qp ^ p and lim n inf i^ En Pi = 00 since inf q p = oo 
by the standard convention. Hence by Theorem 3, the process is unique. As 
we have seen before, Corollary 4 is also applicable in such a trivial case. 

(b) Knowing that the process is unique, then by Theorem 6, there should 
exist a cp satisfying (Ul) / , as well as (U2). The problem is that the resulting p 
is not explicitly known when E is infinite. In this sense, Theorem 6 is theoretic 
correct but not practical in such simplest case. □ 

Example 8 Let E = Z + and Q ^ be a bounded conservative Q-matrix on E. 
Denote its test function by ip ^ = 1 as in the last example. Next, let Q ^ be 
a conservative Q-matrix on E satisfying the assumptions of Theorem 3 with a 
sequence of finite subsets {E n } n ^\ and a test function ip( 2 \ Finally, we construct 
a new Q as follows: on the odd numbers in E, we use the transition mechanism 
of Q^\ and on the even numbers in E, we adopt the one of Q^ 2 \ Define 
p = on the odd numbers and ip = p ^ on the even numbers. Then the 
assumptions of Theorem 3 hold but its test function (p n has no limit as n —> oo: 
linin-xx) ip n = 00 and lim^^ <p n = 1. 

Proof. First, note that for the original Q^ on E, because each E n is a 

( 2 ) 

finite subset of E. the condition lim n _ >00 inf k$E n p\ = 00 is equivalent to 
lim n _»oo 'Pn' 1 = oo* Therefore, we have 

lim p n = lim pff = oo, lim ip n = lim p^ = 1. 

n—»oo n—> oo n^oo n—>co 

To show the assumptions in Theorem 3 hold, simply let Eq = {odd integers}, 
and let E n (n ^ 1) be the union of Eq and the natural modification of the 
original E n used for Q^ 2 \ Then the resulting E n j E as n —* oo, sup keEn < 
oo for each n ^ 0, and 

lim inf pj, = lim inf p[ 2 ^ = lim pffi = oo. 
n —>co k$E n n ->go k$E n n^>co 

Finally, because of the independence of and Q^ 2 \ p^- 1 and p^ 2 \ the 
condition Q(p ^ max{c 2 , l}p on the set of odd numbers follows from 

Q^p 1 ^ ^ p^ on E; 

and the same condition on the set of even numbers follows from 

q( 2 )^( 2 ) ^ C 2 p^ on E. 

We have thus obtained the required conclusion. 

As mentioned in the last proof, in the present situation, we do not know 
how to use Theorem 6. □ 
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Note that the last matrix Q is reducible. However, we can add a connection 
between 0 and 1 to produce an irreducible version of the example. This is not 
essential since a local modification does not interfere the uniqueness problem. 
Furthermore, one may replace the set {odd integers} or {even integers} by any 
infinite subset of E, but not E itself, the set of primer numbers for instance. 
The conclusion of Example 8 remains the same by an obvious modification. 
The point is that some E n is allowed to be infinite in (Ul) but not in (Ul) ; . 

Example 9 The pure birth process is unique iff (2) holds. In particular, set 
q n ,n+ 1 = the nth primer, then Theorem 3 is suitable but Corollary 4 fails. 


Proof. Note that = qk,k+i for k ^ 0. 

( a ) If Hk % 1 = 00 > set E n = {0,1,... ,n} and 

ifik = 1 + y 1 ->oo as k —* co. 

njH i * 

Then Q<p ^ <p and so Theorem 3 gives us the uniqueness of the processes. In 
the particular case that q n , n + 1 = n + 1, the above p> has order logn. However, 
we can also choose <p n = 1 + n and apply Theorem 3. This shows that there 
are some freedom in choosing <p. 

(b) If M := q {T 1 < co, set E n as above and 



V —-Me 

Wk-i q i 



1 

2 


Then sup fc p^ = 1/2 > 0, Q<p ^ (p , and so by Theorem 5, the processes are not 
unique. We remark that it would be awful to use the necessity in Theorems 3 
or 6 to prove this non-uniqueness property. 

(c) The last assertion is due to J.L. Zheng (cf. [3; Example 2.3.12] or [9; 
Example 2.26]). □ 


Proof of the uniqueness for Example 2. For i e E = Z/, define its level 
by 1*1 = Hues anf I se ! E n = {i £ E : |i| ^ n} for n ^ 1. 

(a) Next, define tp(i) = 1 + |i|. Then it is clear that lim^^oo inf k$E n '-p(k) = 
oo. Because the diffusions do not change the levels, we have 

fMI*|) = iK'i'u) - a{i u )\ = X! [«o ~ aii u + a 2 i 2 u - a 3 i^] 

ueS ueS 


for some positive {afc}| =0 - N ex b since 

y *U < I*! 2 ; ji y il > ( J (Jensen’s inequality), 

ueS ' ' ueS 'I '' 
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where \S\ is the candinality of S (finite but arbitrary), we have 
fl<^(|?'|) ^ a' 0 ~ a[ |z| + a’ 2 \i\ 2 — CK 31 z | 3 

for some positive {c>4}| =0 . Now, because the right-hand side becomes negative 
for large enough |i|, it is clear that |z|) ^ op(\i\) for every i e E and large 

enough c. The assertion now follows from Theorem 3. Hopefully, we have 
seen the role played by the geometry of E. The proof shows the power of our 
result. A good sufficiency result may be more effective than a criterion. 

(b) It is also possible to use Corollary 4 to prove the required assertion, 
simply choose p(i) = 7(1 + 2 ueS*«)- Fi rs N choose 7 large enough so that 
p ^ q. Next, choose c large enough so that ftp ^ op. □ 

It is worthy to mention that in accompany to Theorem 3, we also have 
a similar, practical sufficiency result for (exponential) ergodicity. Refer to 
[5; Theorem 3 (GS)], [ 6 ; Theorem (1.18) (DS)], [9; Corollary 4.49 (DS) and 
Theorem 14.1 (GS)]. 

In the past nearly 30 years, Theorem 3 and Corollary 4 have very success¬ 
ful applications. A list of the literature was collected in [10; §9.2], Certainly, 
the results used a lot by the author (in [9] for instance). In particular, it was 
used at the first step to construct a large class of infinite-dimensional pro¬ 
cesses ([9; §13.2]), 15 models are included in [9; §13.4], Corollary 4 with some 
extension was used by Song (1988) [24] in a quite earlier stage for Markov 
decision processes moving from bounded to unbounded situation. It is now 
quite often to see the influence of the study on Markov Jump processes to the 
theory of Markov decision processes. Based on [4], Theorem 3 was collected 
into Anderson [1; Corollary 2.2.16], its originality was unfortunately ignored, 
even though the original paper [4] is included in the references of the book. For 
some corrections and comments on the last book, refer to [7]. Very recently, 
Theorem 3(GS) is applied by Chen and Ma [14] to genetic study having contin¬ 
uous state space. Finally, we mention that the results have already extended 
to the time-inhomogeneous case by Zheng [28] and [27] using the martingale 
approach. 

Before going to the proofs, note that equation (1) is equivalent to 

n(A)tt = u, 0 < a ^ 1 on B, A > 0 , (3) 
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where (pj : j e E) is a positive probability measure on E. The enlarged 
transition probability matrix is irreducible even the original one may be not. 

Lemma 10 The equation (1) has zero solution only iff so does the equation 

n A (A)(ul£:) = u, 0 ^ w ^ 1 on Ea, A > 0. (4) 

Thus, the original Q-process is unique iff the II A (A)-chain is recurrent. 

Proof. Noting that ua = 2 keEPk u k, it is clear that ua = 0 iff = 0 for all 
k e E since Pk > 0 for all k e E. Equation (4) restricted to E coincides with 
(3) and then (1). This proves the first assertion. 

To prove the second assertion, it suffices to note that II A (A)-chain is re¬ 
current iff equation (4) has only trivial solution. The last result comes from 
[26], [3; Lemma 12.1.27], or [9; Lemma 4.51]. We remark here that the regu¬ 
larity assumption used in the cited references can be replaced by the minimal 
process, due to the equivalence of recurrence of the minimal process and its 
embedded chain. Refer to [3; Lemma 12.3.1], or [9; Theorem 4.34], □ 

Proof of Theorem 6. When \E\ < oo, the conclusion is trivial and the 
assumptions hold for the specific E n = E and ^ s 1 as seen from proof (a) 
of Example 7. Hence we may assume that E = Z + . Since each E n is finite, 
the condition lim n inf k$E n Pk = °o becomes lim n _ >00 p n = oo. In this case, 
conditions (Ul) / and (U2) consist a criterion for the recurrence of the Markov 
chain n A (A), refer to [9; Theorem 4.24] and its references within. 

We remark that it is at this point, the finiteness of E n is required and so 
the present sufficiency proof is not suitable for Theorem 3. At the moment, 
we do not know how to extend the necessity result of Theorem 6 from DS to 
GS. 

Here is a part of an alternative proof given in [25]. Let P mm (A) be the 
Laplace transform of P mm (t). Using the second successive approximation 
scheme for the backward Kolmogorov equation (goes back to [17; Theorem 
1 ]), we obtain 

P”“(A) - f; n(A)”diag( r t-) 

n=0 ' "' 

(cf. [9; page 75, line -6]). Hence 

00 

column (1) = ^ n(A) n column 

n =0 

The process is unique iff the left-hand side equals 1 at some/every i e E, the 
right-hand side is the probabilistic decomposition of the time that the Markov 
chain n A (A) starts from some i e E, first visits A at some step n ^ 1, which 
equals 1 iff the irreducible Markov chain n A (A) is recurrent. We have thus 
come back to the last lemma. □ 
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Proof of Theorem 3. Here we adopt a circle argument. 

(Ul)' + (U2) (Ul) + (U2). This is easy since (Ul) is weaker that (Ul)'. 

(Ul) + (U2) => uniqueness. This is the sufficiency part of Theorem 3 and 
was proved long time ago, even for GS. 

Uniqueness ==> (Ul)' + (U2). This is the necessity part of Theorem 6 . □ 

We remark that a similar phenomena is appeared in Theorem 5, the con¬ 
ditions for sufficiency are weaker than the ones for necessity. As we have seen 
from Example 9, this is very helpful in practice. However, these conditions 
are actually equivalent: conditions for necessity ==> conditions for sufficiency 
non-uniqueness => conditions for necessity. 

In view of these discussions, one may combine Theorems 3 and 6 into one 
having the style of Theorem 5. 

In conclusion, this note as well as the practice during the past 30 years 
confirm that the sufficient part of Theorem 3 and Theorem (Criterion) 5 are 
not only powerful but also sharp, even though at the moment we are still 
unable to prove the necessity part of Theorem 3 for general state spaces. 
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